1). Field of the Invention
This invention relates generally to a host computer system and a method for providing results in response to queries, and more specifically to controlling quality of results provided by one or more engines of such a system.
2). Discussion of Related Art
The Internet is often used for obtaining information from search engines and other host computer systems. A user at a live remote computer system enters a live query and transmits the live query over the Internet to such a host (live) computer system. One or more keywords are then extracted from the live query by the host system and entered by the host system into one or more engines. Each engine is optimized to provide a different kind of information such as answering questions, suggesting products to buy, suggesting related web docs, or additional search terms to try (related search). The engines return live results to the host system, and the host system decides how to combine the engine's results into an HTML response page which is transmitted back over the Internet to the live remote computer system for viewing within a browser of the live remote computer system.
The engines are often operated by different teams within an organization and these teams are potentially optimizing for different performance metrics. There is typically no specific common relevance for the functioning of an engine. An engine may for example provide irrelevant results, which may go undetected or unnoticed by a team operating the engine. There is usually also no uniformity in quality control from one engine to the next as different teams may have different standards that do not necessarily correlate with one another and do not provide for the ability to judge the relevance of the page as a whole. System operators may wish to determine the relevance of individual engine results or may wish to evaluate the relevance of the page as a whole and how the results from the various engines are displayed in relation to one another.